Smoke detection method with visual depth

ABSTRACT

The present invention provides a smoke detection method with visual depth, which uses an image camera and a depth camera to extract surrounding images and surrounding depth information. A vehicle is used to patrol an area, such as the area of a processing factory, for receiving the surrounding environment information and detecting existence of burning objects or smoke. Then a processor adopting the clustering algorithm is used to estimate the smoke distribution or the location of fire source of burning objects, and uses an alarm to provide alarm information. Thereby, the rescue crew can prepare in advance and respond immediately. By providing the correct information to the firemen in the fire scenes, the time to control the fire can be shortened and the time for evacuation can be increased.

FIELD OF THE INVENTION

The present invention relates to a smoke detection method with visualdepth, which is a detection method using the clustering algorithm toestimate the smoke distribution of burning objects or the location offire source thereof for providing fire information to rescue crew.

BACKGROUND OF THE INVENTION

The population density in Taiwan is high and thereby the living space islimited. Current residence houses are mostly collective tall buildings.Fires occurring in tall buildings usually lead to serious damages inpeople and belongings. Thereby, complete fire-preventive equipment iscrucial for people living in urban environments with tall buildings.Different fire alarm devices are provided for adapting to the urbanenvironments with dense tall buildings.

The fire alarm device according to the prior art detects by theconcentration of smoke and temperature sensors. The detector and alarmwill not be triggered unless the fire has occurred for a period and thesmoke has accumulated to a certain concentration. The smoke and firedetection systems based on temperature can detect flame and smokeimmediately and launch alarms. Unfortunately, since the fire alarmdevices according to the prior art is fixed, they are limited bydistance and space, making them unsuitable for open spaces, areas withmany corners, or outdoor spaces. In addition, the fire alarm devicesaccording to the prior art cannot provide the information such as thelocation of fire and the smoke distribution when a fire occurs. Besides,the real-time fire detector based on temperature might have the problemof false alarms.

A fire detection device combined with a camera can acquire morefire-related information real-timely from the monitor. Compared with thesmoke detection and temperature sensing devices, it can detect stablyand response rapidly. Thereby, vision-based fire and smoke detectiondevices have become an important development direction for researchingand designing detection devices for fire prevention.

Accordingly, the present invention provides a smoke detection methodwith visual depth, which uses an image camera and a depth camera tophotograph the fire scene for acquiring surrounding images and depthimages. Then a processor adopts the clustering algorithm to distinguishthe smoke distribution and the location of fire source of the burningobjects. By illustrating the fire scene completely, the rescue crew canrespond promptly and the correct information can be provided to thefiremen in the fire scene. By shortening the time to extinguish thefire, the survival opportunity of people can be improved.

SUMMARY

An objective of the present invention is to provide a smoke detectionmethod with visual depth, which uses an image camera and a depth camerato photograph the fire scene for acquiring surrounding images and depthimages. Then a processor adopts the clustering algorithm to estimate thesmoke distribution and the location of fire source of one or moreburning object in the surrounding images.

To achieve the above objective and effect, the present inventionprovides a smoke detection method with visual depth, which comprisessteps of: extracting a plurality of surrounding images according to oneor more image camera of a vehicle, and acquiring a plurality pieces ofsurrounding depth information of the plurality of surrounding imagesaccording to one or more depth camera of the vehicle; identifying one ormore burning object in the plurality of surrounding images located onone side of the vehicle according to a processor; acquiring imageinformation and an outline of the one or more burning object accordingto the plurality of surrounding images and the plurality pieces ofsurrounding depth information; estimating the smoke distribution or alocation of fire source of the one or more burning object according tothe image information and the outline of the one or more burning objectand according to a clustering algorithm; and producing alarm informationaccording to the smoke distribution or the location of fire source ofthe one or more burning object.

According to an embodiment of the present invention, in the step ofjudging if the plurality of surrounding images including one or moreburning object located on one side of the vehicle according to aprocessor, the one or more burning object includes smoke and a firesource.

According to an embodiment of the present invention, the one or moredepth camera includes a structured-light projection module and astructured-light camera. The structured-light projection module projectsa plurality of light planes to the one or more burning object. Thestructured-light camera receives the projections of the plurality oflight planes and reflects a light-image message for acquiring theplurality pieces of surrounding depth information.

According to an embodiment of the present invention, thestructured-light projection module includes a laser light-emittingdevice and a lens set.

According to an embodiment of the present invention, in the step ofproducing alarm information according to the smoke distribution or thelocation of fire source of the one or more burning object, one or moreaudio unit of the alarm module is adopted to launch the alarminformation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a flowchart according to an embodiment of the presentinvention;

FIG. 2 shows a structural schematic diagram according to an embodimentof the present invention;

FIG. 3 shows a schematic diagram of binarization threshold valueaccording to an embodiment of the present invention; and

FIG. 4 shows a schematic diagram of detecting flame according to thepresent invention.

DETAILED DESCRIPTION

In order to make the structure and characteristics as well as theeffectiveness of the present invention to be further understood andrecognized, the detailed description of the present invention isprovided as follows along with embodiments and accompanying figures.

The present invention provides a smoke detection method with visualdepth, which uses a processor adopting the clustering algorithm toestimate the smoke distribution or the location of fire source ofburning objects, and uses an alarm to provide alarm information.Thereby, the rescue crew can prepare in advance and respond immediately.By providing the correct information to the firemen in the fire scenes,the time to control the fire can be shortened and the time forevacuation can be increased.

Please refer to FIG. 1, which shows a flowchart according to anembodiment of the present invention. As shown in the figure, the smokedetection method with visual depth according to the present embodimentcomprises the following steps:

-   Step S10: Extracting surrounding images according to the image    camera of a vehicle, and acquiring the surrounding depth information    of the surrounding images according to the depth camera of the    vehicle;-   Step S20: Identifying the burning object in the plurality of    surrounding images located on one side of the vehicle according to a    processor;-   Step S30: Acquiring image information and an outline of the burning    object according to the surrounding images and the surrounding depth    information;-   Step S40: Estimating the smoke distribution or a location of fire    source of the burning object according to the image information and    the outline of the burning object and according to a clustering    algorithm; and-   Step S50: Producing alarm information according to the smoke    distribution or the location of fire source of the burning object.

Please refer again to FIG. 1 and FIG. 2. FIG. 2 shows a structuralschematic diagram according to an embodiment of the present invention.As shown in the figures, in the step S10, one or more image camera 10 ofa vehicle is used to photograph the surrounding for extracting aplurality of surrounding images; and one or more depth camera 20 of thevehicle 1 is used to acquire a plurality pieces of surrounding depthinformation of the plurality of surrounding images. The surroundingdepth can refer to the distance between the depth camera 20 and varioussurrounding locations. According to an embodiment, the vehicle 1 can bea mobile vehicle including unmanned mobile vehicle, for example,unmanned aerial vehicles unmanned ground vehicles. The image camera 10includes image sensors such as CCD or CMOS sensors. In addition, theimage camera 10 can include night vision devices or active infraredlighting for supporting night vision functions.

Please refer to FIGS. 1 to 3. FIG. 3 shows a schematic diagram ofbinarization threshold value according to an embodiment of the presentinvention. In the step S20, the processor 30 adopts dilation and erosionto binarize the images in the HSV color space. The acquired plurality ofsurrounding images are analyzed for identifying the target:m=Σ _(i≤1) ^(n) f (x,y)  Equation 1where m is the binarization threshold value; f is the input image; n isall pixel items; and f(x,y) is the greyscale value of the pixelcoordinates. By using the above Equation 1, the one or more burningobject 2 located on one side of the vehicle 1 in the plurality ofsurrounding images can be identified.

The processor 30 is coupled to the image camera 10 and one or more depthcamera 20. The one or more depth camera 20 includes a structured-lightprojection module 22 and a structured-light camera 21. When the imagecamera 10 extracts a plurality of surrounding images, thestructured-light projection module 22 projects a plurality of lightplanes 23 to the surface of the one or more burning objectcorrespondingly. Then the structured-light camera 21 receives alight-image message reflected from the projection of the plurality ofsurrounding images for acquiring a plurality pieces of surrounding depthinformation of the plurality of surrounding images. The detection methodaccording to the present embodiment is to use the structured-lightprojection module 22. The principle is that a light source is used toproject controllable light spots, stripes, or planes on the surface ofthe object under test. Then a sensor such as a camera is used to capturethe reflected image. After geometric calculations, the stereoscopiccoordinates of the object will be given. According to a preferredembodiment, the structured-light projection module 22 adopts a laserlight-emitting device 221 as the light source. Laser is superior togeneral light sources in its high coherence, low attenuation, longmeasurement distance, high accuracy, as well as low vulnerability to theinfluence by other light sources. The laser provided by the laserlight-emitting device 221 is dispersed by a lens set 222 to form a lightplane 23 in the space. According to a preferred embodiment, the lens set222 can include a pattern lens, which includes patterned microstructuresto enable the light plane 23 formed by the transmissive laser to ownpattern features, such as light-spot matrix on a two-dimensional plane.

Please refer to FIGS. 1 and 2. In the step S30, the processor 30, whichcan be a field-programmable gate array (FPGA), calculates and analyzesthe plurality of surrounding images and the plurality pieces ofsurrounding depth information. By using the variation, the distancebetween the one or more burning object 2 and the vehicle 1, and theimage information and an outline of the one or more burning object willbe given.

Please refer to FIGS. 1 and 2. In the step S40, estimate the smokedistribution or a location of fire source of the one or more burningobject 2 according to the image information and the outline of the oneor more burning object 2 and according to a clustering algorithm. Theclustering algorithm is to use the plurality pieces of depth informationto give the image formation of the one or more burning object 2. Assumethat the sample number m is {x¹, x², x³, . . . x^(m)}, x^(i) ∈ R^(n),where R^(n) is the statistical reference model. The K clustering centersselected randomly from the sample set i is {μ₁, μ₂, μ₃, . . . . μ_(k)},μ_(j) ∈ R^(n). Calculate the cluster center for each sample set i togive:C ^(i)=1˜K  Equation 2C ^(i)=argmin ∥x ^(i)−μ_(j)∥²  Equation 3where K is the number of clusters; C^(i) is the classification of thesample set i closest to the cluster K; μ_(j) is the estimated locationof the first cluster center; and j is the center of mass of each clustercenter. According to Equation 3:

$\begin{matrix}{\mu_{j} = \frac{\sum_{i = 1}^{m}{\left\{ {c^{i} = j} \right\} x^{i}}}{\sum_{i = 1}^{m}\left\{ {c^{i} = j} \right\}}} & {{Equation}\mspace{14mu} 4}\end{matrix}$

According to Equation 4, reducing a point of the cluster K gives thefollowing equations:

$\begin{matrix}{{\mu_{j} = \frac{x^{1},x^{2},x^{3},{\ldots\mspace{14mu} x^{m}}}{x^{i}}},{\overset{\prime}{\mu_{j}} = \frac{x^{1},x^{2},x^{3},{\ldots\mspace{14mu} x^{m}}}{x^{i} - 1}}} & {{Equation}\mspace{14mu} 5} \\{\overset{\prime}{\mu_{j}} = \frac{{\mu_{j}x^{i}} - x^{m}}{x^{i} - 1}} & {{Equation}\mspace{14mu} 6} \\{\overset{\prime}{\mu_{j}} = \frac{{\mu_{j}\left( {x^{i} - 1} \right)} + \mu_{j} - x^{m}}{x^{i} - 1}} & {{Equation}\mspace{14mu} 7} \\{\overset{\prime}{\mu_{j}} = {\mu_{j} + \frac{\mu_{j} - x^{m}}{x^{i} - 1}}} & {{Equation}\mspace{14mu} 8}\end{matrix}$where {acute over (μ)}_(j) is the estimated location of the secondcluster center.

By using the clustering algorithm as described above, the fire sourceand the diffusion and distribution of smoke can be given rapidly. Thenthe information of on-site situation can be delivered rapidly to relatedstaff for judgement. After judgement, the on-site firemen can beinformed with the coping method for cutting the spreading direction offire and dispelling smoke early. Thereby, the efficiency of rescue andextinguishment of fire can be enhanced.

Please refer to FIGS. 1 and 2. In the step S50, the processor 30 iscoupled to an alarm module 40, which includes one or more audio unit 41.The alarm information is produced according to the smoke distribution orthe locations of fire source of the one or more burning object andprovided to the one or more audio unit 41 for launching alarm such asalarm sound. According to the present embodiment, the alarm module 40can be connected wirelessly to remote devices via various wirelesscommunication methods such as Wi-Fi, 3G, 4G, 5G, Bluetooth. In addition,the information is disposed on a display 70 coupled with the processor30 for providing the information of the fire scene.

Please refer again to FIG. 2. As shown in the figure, the processor 30is coupled to a database 50, which is used for storing the plurality ofsurrounding images and the plurality pieces of surrounding depthinformation estimated by the processor 30 and extracted by the imagecamera 10 and the depth camera 20. According to the present embodiment,the processor 30 is further coupled to a sensor 60, which can sense themoving state of the vehicle 1. Thereby, the moving speed of the one ormore burning object 1 can be estimated for prevent the vehicle 1 frombumping into the one or more burning object 1.

According to a preferred embodiment, a power supply 80 is disposed onthe vehicle 1 and connected electrically to the image camera 10, thestructured-light unit 21 and the laser light-emitting device 221 of thedepth camera 20, the alarm module 40, and the display 70 for supplyingpower source.

Please refer to FIG. 4, which shows a schematic diagram of detectingflame according to the present invention. As shown in the figure, thesurrounding image includes one or more burning object 2, which includessmoke 5 and a fire source 3. According to the present embodiment, theprocessor 30 estimates the smoke distribution 6 of the smoke 5 and thelocation 4 of the fire source. The vehicle 1 is used to patrol a regionand detect the locations of the first source and the smoke as well asthe spreading direction of the smoke. By using the clustering algorithm,the concentration portion of the fire source can be identified. Then thealarm is launched to the rescue crew for coping with the situation.

To sum up, the present invention provides a smoke detection method withvisual depth, which mainly uses the image camera and the depth camera toform dual lenses for photographing the fire scene. The images aretransmitted to the surveillance crew via Wi-Fi network. Then the firescene can be seen remotely in the smoke detection system and providingthe following benefits:

-   1. Classify the flame and smoke using the image module with visual    depth, so that the rescue crew can know the severity level of fire    and extinguish the fire rapidly. In addition, by using the    clustering algorithm, the locations of the smoke and the fire source    can be identified. Thereby, the rescue crew can put out the fire    promptly and inform the trapped people to evacuate early for    reducing casualties.-   2. The vehicle is used to illustrate the factory scene on the remote    device of the staff. When a disaster occurs, the firemen are    informed immediately for judging and handling promptly according the    fire scene. Compared with the past, in which assessment and action    cannot be started before reaching the fire scene, the present    invention can improve the response efficiency for fire as well as    increasing the rescue efficiency.

Accordingly, the present invention conforms to the legal requirementsowing to its novelty, nonobviousness, and utility. However, theforegoing description is only embodiments of the present invention, notused to limit the scope and range of the present invention. Thoseequivalent changes or modifications made according to the shape,structure, feature, or spirit described in the claims of the presentinvention are included in the appended claims of the present invention.

What is claimed is:
 1. A smoke detection method with visual depth,comprising steps of: providing an image camera and a depth camera;extracting a plurality of surrounding images according to said imagecamera, and acquiring a plurality pieces of surrounding depthinformation of said plurality of surrounding images according to saiddepth camera; identifying one or more burning object in said pluralityof surrounding images according to a processor; acquiring imageinformation and an outline of said one or more burning object accordingto said plurality of surrounding images and said plurality pieces ofsurrounding depth information; estimating the smoke distribution of saidone or more burning object or a location of fire source thereofaccording to said image information of said one or more burning objectand said outline thereof and according to a clustering algorithm; andproducing alarm information according to the smoke distribution of saidone or more burning object or the location of fire source thereof. 2.The smoke detection method with visual depth of claim 1, wherein saidstep of identifying one or more burning object in said plurality ofsurrounding images according to the processor, said one or more burningobject includes smoke and a fire source.
 3. The smoke detection methodwith visual depth of claim 1, wherein said one or more depth cameraincludes a structured-light projection module and a structured-lightcamera; said structured-light projection module projects a plurality oflight planes to said one or more burning object; said structured-lightcamera receives a light-image message reflected by the projection ofsaid plurality of light plane for acquiring the plurality pieces ofsurrounding depth information.
 4. The smoke detection method with visualdepth of claim 3, wherein said structured-light projection moduleincludes a laser light-emitting device and a lens set.
 5. The smokedetection method with visual depth of claim 1, wherein said step ofproducing alarm information according to the smoke distribution of saidone or more burning object or the location of fire source thereof, oneor more audio unit of said alarm module is used to launch said alarminformation.
 6. The smoke detection method with visual depth of claim 1,wherein said clustering algorithm adopts the equations$\mu_{j} = \frac{\sum_{i = 1}^{m}{\left\{ {c^{i} = j} \right\} x^{i}}}{\sum_{i = 1}^{m}\left\{ {c^{i} = j} \right\}}$and C^(i)=argmin ∥x^(i)−μ_(j)∥²; K is the number of clusters; C^(i) isthe classification of the sample set i closest to the cluster K; μ_(j)is the estimated location of the first cluster center; and j is thecenter of mass of each cluster center.
 7. The smoke detection methodwith visual depth of claim 1, wherein said image camera or said depthcamera is installed on a vehicle.
 8. The smoke detection method withvisual depth of claim 7, wherein said vehicle includes a mobile vehicle.9. A smoke detection method with visual depth, comprising steps of:providing an image camera and a depth camera; extracting a plurality ofsurrounding images according to said image camera, and acquiring aplurality pieces of surrounding depth information of said plurality ofsurrounding images according to said depth camera; acquiring imageinformation and an outline of one or more burning object according tosaid plurality of surrounding images and said plurality pieces ofsurrounding depth information; estimating the smoke distribution or alocation of fire source of said one or more burning object according tosaid image information of said one or more burning object and saidoutline thereof and according to a clustering algorithm; and producingalarm information according to the smoke distribution of said one ormore burning object or the location of fire source thereof; where saidclustering algorithm adopts the equations$\mu_{j} = \frac{\sum_{i = 1}^{m}{\left\{ {c^{i} = j} \right\} x^{i}}}{\sum_{i = 1}^{m}\left\{ {c^{i} = j} \right\}}$and C^(i)=argmin ∥x^(i)−μ_(j)∥²; K is the number of clusters; C^(i) isthe classification of the sample set i closest to the cluster K; μ_(j)is the estimated location of the first cluster center; and j is thecenter of mass of each cluster center.
 10. The smoke detection methodwith visual depth of claim 9, further comprising a step of identifyingsaid one or more burning object in said plurality of surrounding imagesaccording to a processor.